Object location apparatus

ABSTRACT

A translucent support, which may be the support surface of an article conveyor, supports objects, locations or dimensions of which are to be determined. A point light source or light sources severally illuminate the surface and object thereupon. An imaging apparatus located below the translucent surface generates signals representing the images of the illuminated translucent surface and the shadows generated by the object thereon. The signals for two or more illumination conditions are processed to determine the locations of at least portions of the object. A particular avatar of the invention uses a conveyor with translucent web.

The Government has rights in this invention pursuant to contract No.104230-84-D-0929 with the Post Office.

BACKGROUND OF THE INVENTION

Modern production systems require automation for handling objects, asfor moving work pieces from one part of a factory to another, forfeeding work pieces into a machine for manufacturing operations and forremoving the resulting processed workpieces, for packing finishedproducts and the like. In many contexts, the objects are identical,whereby specialized sorting, orientation and gripping devices may beused. In some contexts, the sequence involves objects which are notidentical. For example, certain automobile manufacturers currently haveproduction lines in which different models alternate, or in whichdifferent models may be randomly placed along the production line. Forsuch situations, object identification and location schemes arenecessary.

Among the more difficult problems in article sorting and processing areidentification of the size and shape of the object, and identificationof its location. U.S. Pat. No. 4,063,820 issued Dec. 20, 1977 to Borgesedescribes a tilted conveyor belt in which parcels gravitate toparticular locations on the belt, together with a system of narrow lightbeams which are broken by the edges of the advancing parcel. In theBorgese arrangement, breaking a light beam causes the signal generatedby the associated photosensor to change. Since the locations of thephototransmitter which generates each light beam and its associatedphotosensor are known, geometrical relationships may be used todetermine the dimensions of the parcel, and its location may also bedetermined because the speed of the conveyor belt is fixed. Once thesize and shape of the object are determined, and its location isestablished, a gripper may be used to grasp it and to place the objectfor further processing. The Borgese arrangement requires movement of theobject to break the various light beams.

In the context of determining the size, shape and location of elongatedobjects such as in the cutting of boards from logs, U.S. Pat. No.4,803,371 issued Feb. 7, 1989 to Durland describes a scheme in which theboards are supported by a conveyor at locations spaced along theirlengths, and are simultaneously illuminated from above and below byscans of light. Cameras placed above and below the boards view theboards, discriminate the light, and process the resulting image signalsto generate signals representative of the dimensions of the board. Thisarrangement suffers from the disadvantages that objects of arbitrarysize and shape cannot be conveyed and processed in a similar manner, andthat the light must be scanned. An object location scheme usable forrandomly sized objects is desired, in which relative motion of theobject and light is not mandatory.

SUMMARY OF THE INVENTION

An apparatus according to the invention includes a translucent ordiffusing support surface, which may be fixed or which may be a part ofa conveyor, which surface is illuminated by at least first and secondnondiffuse light or other radiation sources. Each source causes anopaque object located on the support surface to cast a shadow onto thetranslucent surface. The translucent surface is viewed from below by animager or camera to form signals representative of the images ofilluminated and shadowed portions of the support surface. The signalsare processed to determine a dimension or a location of at least aportion of the object In a particular avatar or embodiment of theinvention, a diffuse light source may be used to cast a shadow showingan outline of the object.

DESCRIPTION OF THE DRAWING

FIG. 1 is a perspective or isometric view of a translucent conveyor beltfor conveying objects, together with diffuse and nondiffuse lightsources, an imaging apparatus, and a control and processing arrangement;

FIGS. 2a and 2b, referred to jointly as FIG. 2, are schematicrepresentations of illumination of an object by the sources of FIG. 1under various conditions of energization;

FIG. 3a illustrates illumination of an object by a pair of sourcesarrayed in different locations than those of FIGS. 1 and 2, and FIGS. 3band 3c illustrate the shadows seen from below under two differentillumination conditions; FIGS. 3a, 3b and 3c are jointly referred to asFIG. 3;

FIG. 4 is a simplified block diagram of a processing arrangement forsignals representing the images of FIGS. 3b and 3c; and

FIG. 5 is a simplified flow chart illustrating processing fordetermining the unknown location and height of the right edge of theobject in FIG. 3; and

FIGS. 6a and 6b illustrate conceptual steps of signal processing.

DESCRIPTION OF THE INVENTION

FIG. 1 is a perspective or isometric view of an avatar of the inventionfor determining the location of at least a portion of an object. In FIG.1, an endless conveyor belt 10 is translucent to visible light. Conveyorbelt 10 is supported by four toothed drive cylinders 12a, 12b, 12c and12d. The conveyor belt 10 is driven in the direction of arrow 14 by amotor 16 mechanically coupled to drive cylinder 12d. Conveyor belt 10may be made from clear acrylic material which has been renderedtranslucent by roughening or etching of the surface. A feed conveyorapparatus designated generally as 11 dumps unsorted, random-size objectssuch as parcels onto the left end of conveyor belt 10. The objectstravel to the right side of the conveyor, where they may be picked up bya gripper, illustrated as 98, carried on a robot arm 96.

As illustrated in FIG. 1, an object having a shape illustrated as aparallelepiped 20 is supported by the upper side of conveyor belt 10 andis transported in the direction of arrow 14. A first non-diffuse lightsource 22 is placed at a known location above the plane in which theupper portion of conveyor belt 10 lies. A similar non-diffuse lightsource 24 is placed at another location separated from the location oflight source 22. Light sources 22 and 24 may be point light sources suchas arc lamps or small lamps approximating a point source, or they may bescanned laser sources, all as known in the art. A diffuse light sourceillustrated as a fluorescent lamp 26 together with a planar diffuser 27is located generally above conveyor belt 10 to illuminate the region ofconveyor belt 10 in which objects are to be located. Light sources 22,24 and 26 are coupled to a control arrangement illustrated as block 30.As described below, block 30 controls the energization of light sources22, 24 and 26, and also processes certain signals.

When light source 22 is energized and sources 24 and 26 are deenergized,source 22 illuminates the upper side of conveyor belt 10. Certain rays,illustrated as 32, extend between point source 22 and an edge 34 ofobject 20. The projection of rays 32 onto translucent surface 10 definesthe limits of a shadow, a portion which is illustrated as 36, and theremainder of which extends under object 20 Similarly, when light source22 is deenergized, but point light source 24 is energized, the surfaceof conveyor belt 10 and object 20 are illuminated. Certain light raysillustrated as 38 extend from point source 24 past a further edge 40 ofobject 20. The projection of rays 38 onto the upper side of conveyorbelt 10 define the limits of a further shadow illustrated as 42, theremainder of which extends under object 20. When diffuse light source26, 27 is energized and point light sources 22 and 24 are deenergized,the entire upper side of conveyor belt 10 is illuminated, except for ashadowed portion lying directly under object 20.

A television camera illustrated as 44 is located between the upper andlower portions of conveyor 10, and includes a lens arrangement 46 forviewing the underside of the upper portion of conveyor belt 10. Camera44 is supported by a cantilever arrangement 48. Camera 44 is energizedand controlled from control arrangement 30 over a multiconductor cable50, which also provides a data path by which signals representative ofimages may be coupled back to control arrangement 30.

In operation, conveyor 11 dumps objects onto conveyor 10, which carriesthem to the region over camera 44. Control block 30 alternatelyenergizes sources 22, 24 and 26, to cast different shadow patterns ontotranslucent conveyor 10. Camera 44 views the images of illuminated andshadowed regions of conveyor belt 10 under the various conditions ofillumination, to generate signals representative of the images, fromwhich information the locations of at least portions of objects placedon conveyor belt 10 may be determined by triangulation and othercalculations based upon the known locations of the sources and conveyor.It is assumed herein that the conveyor speed is slow enough so that theprocessing of data for location of the object is essentiallyinstantaneous, so that the object is essentially stationary during themeasurement. If this should not be the case, the conveyor may be stoppedor slowed during computation so that the location of each portion of theobject may be more accurately determined. The signals representative ofthe position, size, etc. of the object are sent over a data path 94 to autilization device (not illustrated) which may for example be associatedwith a controller for robot arm 98 and gripper 98, for grasping theobject and for moving it to another location.

It should particularly be noted that the invention is particularly welladapted for identifying dimensions of objects supported on a stationarysupport, corresponding to the condition in FIG. 1 in which conveyor belt10 is a translucent tabletop. The principles of the invention are thesame for a stationary or essentially stationary object.

FIG. 2a is a simplified side or elevation view of a portion of thearrangement of FIG. 1, illustrating geometrical considerations which maybe taken into account in processing shadow information in accordancewith the invention. Elements of FIG. 2 corresponding to those of FIG. 1are designated by the same reference numerals. As illustrated in FIG.2a, source 22 is at a height H in the z direction above the uppersurface of conveyor belt 10, and sources 22 and 24 are at the sameheight and spaced D apart in the x direction. Camera 44 is aligned withthe physical coordinate system, as described below. The camera'sposition, optics and scan format together establish a simple linearrelationship between the camera's image coordinates (i,j) and thephysical coordinates (x,y) of the shadow-images or the translucentsurfaces. In FIG. 2a, source 22 is energized and source 24 isdeenergized. Source 22 produces light, portions of which are illustratedby rays 32 and 54.

As illustrated in FIG. 2a, ray 32 grazes edge 34 of object 20, and castsa shadow which ends at a point x_(a). The shadow extends from x_(a) tothe left and under object 20, to the intersection at position x_(L) ofray 54 with the bottom left corner of object 20.

FIG. 2b is similar to FIG. 2a, but with source 24 energized and source22d energized. Object 20 casts a shadow delimited by the projection ofray 38 grazing edge 40, and extending to intersect conveyor 10 atposition x_(b). The shadow extends to the right from x_(b), under object10, to the intersection at position x_(r) of ray 58 with the lower rightcorner of object 40.

As illustrated in FIG. 2a, when light source 22 is energized, the leftx-coordinate x_(l) can be determined directly from the left-most darkarea which is imaged. The position of the shadow cast by the right edgeat location x_(a) is stored in the form of data. When source 22 isdeenergized and source 24 is energized, as illustrated in FIG. 2b, theposition of the lower right edge of object 20 is directly determined asbeing at x_(r). The shadow position x_(b) is noted. The z positions ofedges 34 and 40 can also be determined from this information. The heightz_(L) of left edge 40 and z_(R) of right edge 34 are determined,respectively, as

    z.sub.L =H(x.sub.a -x.sub.R)/((D/2)+x.sub.a)               (1)

    .sub.zR =H(x.sub.b -x.sub.L)/((D/2)+x.sub.b)               (2)

where sources 22 and 24 are located above x-positions--D/2 and +D/2,respectively.

The physical coordinates (x, y) of points on translucent surface 10corresponding to points of interest (i,j) in the camera image can bedetermined by

    y=a.sub.1 * i+b.sub.1

    x=a.sub.2 * j+b.sub.2

where the * represents multiplication, and where a₁, a₂, b₁, b₂ are thelinear parameters of the camera imaging system as determined by simplegeometrical relations from the known camera position, optics, andscanning parameters. Such geometrical representations are very simple toimplement in a processor, and may readily be extended to objects withmore complicated geometries and positioning.

FIG. 3a is similar to FIG. 2a, and corresponding elements are designatedby the same reference numerals. FIG. 3a illustrates point sources 22 and64 spaced H above the upper surface of conveyor belt 10 and spaced Dapart, but unlike FIG. 2a, object 20 does not lie between the lightsources. Object 20 is at a location which is downstream, in thedirection of arrow 14, relative to sources 22 and 64. In this figure, xais the shadow edge generated by light source 22, xb is the shadow edgegenerated by light source 64, and xR and zR are the unknown right shadowedge and height. If the x-axis origin is located under light source 22,the solutions for these unknowns are ##EQU1##

FIG. 3b illustrates a shadow image having width in the i direction andlength in the j direction, which represents the image seen by camera 44in FIG. 3a when light source 64 is energized and light source 22 isdeenergized. The vertical i axis is at right angles to or crosses thecamera scan lines, and the j axis is parallel to the scan lines, so thei axis may be considered to be the scan line number axis, and the j axisis the time index or sample number along a scan line. FIG. 3cillustrates the image of the corresponding shadow cast when source 22 isenergized and source 64 deenergized. As illustrated in FIG. 3c, theshadow has width W and length L₂.

FIG. 4 illustrates a block diagram of equipment associated with controlcircuit 30 for processing signals representative of the images of FIGS.3b and 3c. Elements of FIG. 4 corresponding to those of FIG. 1 aredesignated by the same reference numerals. In FIG. 4, cable 50 carriesscanned image information to a threshold circuit illustrated as a block410. Threshold circuit 410 generates a binary (two-level) signalrepresentative of the presence or absence of shadow along the camerascan line (j) direction. The binary signal is coupled by way of a datapath 412 to a clocked sample-and-hold (S/H) circuit 414, which breaksthe information into discrete samples. The samples are applied over adata path 416 to a multiplex (MPX) circuit illustrated as a block 418.

Multiplex circuit 418 is controlled by way of signals applied from aprogrammer illustrated as a block 428. Programmer 428 also controls theenergization of sources 22 and 24 of FIG. 1. When source 22 isenergized, a particular shadow pattern is generated, which may besimilar to that of FIG. 3b. The camera image is converted to binary formin blocks 410 and 414 of FIG. 4, and MPX circuit 418 is controlled tocouple the resulting signals B₁ (i,j) to a binary image buffer 424.Buffer 424 stores the frame of signals representing the shadows createdby energization of source 22. When this is completed, programmer 428deenergizes source 22 and energizes source 24, thereby creating a secondshadow pattern, such as that of FIG. 3c. At the same time, programmer428 switches multiplex circuit 418 to couple the resulting binarysignals B₂ (i,j) to a binary image buffer 426, where they are stored.The stored shadow images are available on read data paths 432 and 434for further processing to determine object dimensions.

FIG. 5 is a flow chart of the processing used with the arrangement shownin FIG. 3 to determine the height and location of the right-most edgesof objects. In FIG. 5, the logic flow starts at a START block 510 andproceeds to a cascade of blocks 512, 514 and 516, which togetherrepresent the formation of a difference (Δ) image. FIG. 6a represents as610 the shadow produced during energization of source 22, as imaged bycamera 44 and placed in binary form by the arrangement of FIG. 5.Dash-line 612 represents the corresponding shadow during energization ofsource 24. FIG. 6b represents the difference between the shadowsrepresented by the signals stored in buffers 424 and 426 of FIG. 4. Inparticular, block 512 represents reading of buffer 424 to accessinformation relating to shadow 610 of FIG. 6a, block 514 representsreading of buffer 426 to access information relating to shadow 612 ofFIG. 6a, and block 516 represents the equivalent of an exclusive-or(EXOR) operation performed on the stream of data ##EQU2## whichconceptually generates the difference image 614 of FIG. 6b.

Block 518 represents location of the first positive-going transition ofeach scan j of the Δ image of FIG. 6b, which for each scan line definesa location Tb. Similarly, block 520 represents location of the firstnegative-going transition, which for each scan line defines a locationTa. As illustrated in FIG. 6, the shadows are such that all scan lineshave the same value of Ta and Tb, but this is not necessary to theinvention.

Block 522 represents conversion of Ta and Tb in the i, j coordinate tophysical coordinates x, y,

    x.sub.b (i)=a.sub.2 *T.sub.b (i)+b.sub.2                   (7)

    x.sub.a (i)=a.sub.2 *T.sub.a (i)+b.sub.2                   (8)

Block 524 represents calculation of the x- and z- locations of the edgesof the object ##EQU3##

Other embodiments of the invention will be apparent to those skilled inthe art. In particular, light sources having different spectraldistributions may be used in conjunction with cameras sensitive to thedifferent components, so that the object may be illuminatedsimultaneously by several sources to cast shadows having differentspectral compositions, to thereby obviate the delay attributable tosequential energization of the light sources. While visible lightsources have been described, electromagnetic radiation which does notlie in the visible light band may be used either alone or in conjunctionwith visible light, so long as the support surface is translucent to atleast some of the radiation or light. Fixed translucent support surfacessuch as tables may be used where appropriate, rather than the describedmovable conveyor surface. While sources such as 64 of FIG. 3a have beendescribed as point sources, they are point sources only in theprojection of the FIGURE, and may therefore be line sources extendingperpendicular to the plane of the FIGURE. A location of a single edgemay be identified by the use of a one-dimensional single line scancamera, if desired.

What is claimed is:
 1. An apparatus adapted for receiving an object andfor determining the location of at least one portion of the object, saidapparatus comprising:a support surface including first and second sides,and adapted for supporting objects on said first side of said surface;at least non-diffuse first and second sources of electromagneticradiation, the wavelength of which is selected so that said supportsurface is translucent thereto and said object is expected to be opaquethereto, said first and second sources being located at predeterminedfirst and second separate positions, spaced away from and directingradiation toward said first side of said surface, whereby, if an objectis present on said support surface, at least one shadow of said objectis cast upon said surface by said object; imaging means spaced away fromsaid second side of said support surface for viewing said second sideand for generating image signals representing illuminated and shadowedportions of said second side of said support surface; memory means forstoring memory signals representative of at least the relative locationsof said first and second sources, said support surface and said imagingmeans; and signal processing means coupled to said imaging means and tosaid memory means for processing said image signals and said memorysignals and for generating signals representative of the location of atleast one portion of an object located on said support surface.
 2. Anapparatus according to claim 1, wherein said memory means is apreprogrammed ROM.
 3. An apparatus according to claim 1, wherein saidsupport surface is flat.
 4. An apparatus according to claim 1, whereinsaid support surface is associated with conveyor means.
 5. An apparatusaccording to claim 1, wherein said signal processing means furthercomprises control means coupled to said first and second sources forenergizing said sources as part of a sequence for thereby illuminatingat least one of said objects which may be on said support surface insaid sequence, whereby shadows are cast differently during saidsequence.
 6. An apparatus according to claim 5, wherein said signalprocessing means further comprises image memory means coupled to saidimaging means for separately storing, during at least a portion of saidsequence, information relative to the images of said shadows.
 7. Anapparatus according to claim 6 wherein said image memory means storesinformation relating to the image of least one said shadow during a stepof said sequence in which said first source is energized, and separatelystores information relating to the image of at least one said shadowduring a step of said sequence in which said second source is energized.8. An apparatus according to claim 6 wherein said signal processingmeans further comprises differencing means coupled to said image memorymeans for taking the difference between at least some portions of saidseparately stored information.
 9. An apparatus according to claim 8wherein said signal processing means further comprises counting meanscoupled to said differencing means for determining a dimension of atleast one said difference.
 10. An apparatus according to claim 1 whereinsaid non-diffuse source is a point source.
 11. An apparatus according toclaim 1, wherein said sources are sources of visible light, and saidsupport surface is translucent, whereby ordinary objects such as parcelsare opaque.
 12. An apparatus according to claim 10 further comprising adiffuse light source spaced away from said first side of said supportsurface for illuminating said first surface and any object locatedthereupon, whereby an outline shadow of said object appears on saidsecond side of said support surface.
 13. A method for determining thelocation of at least one portion of an object, comprising the stepsof:placing said object on a translucent surface; illuminating saidobject by means of a non-diffuse light source placed at a known firstlocation spaced away from the side of said surface upon which saidobject is placed; illuminating said object by means of a non-diffuselight source placed at a known second location spaced away from saidside of said surface upon which said object is placed; imaging that sideof said translucent surface opposite to said side upon which said objectis placed by means of imaging means for generating signalsrepresentative of illuminated and shadowed portions of said oppositeside of said surface; and calculating the position of at least oneportion of said object by processing at least said signalsrepresentative of illuminated and shadowed portions of said oppositeside of said surface.
 14. A method according to claim 13, furthercomprising the step of storing at least said signals representative ofilluminated and shadowed portions generated during said step ofilluminating said object by means of a light source placed at said firstlocation.
 15. A method according to claim 14 further comprising the stepof taking the difference between said signals representative ofilluminated and shadowed portions generated during said steps ofilluminating.
 16. A method according to claim 13 further comprising thestep of performing said steps of illumination in time sequence.
 17. Amethod according to claim 13 further comprising the step of illuminatingsaid object by diffuse light to thereby project an outline image to saidother side of said translucent surface.